A Systematic Mapping Study of Code Quality in Education -- with Complete Bibliography

While functionality and correctness of code has traditionally been the main focus of computing educators, quality aspects of code are getting increasingly more attention. High-quality code contributes to the maintainability of software systems, and should therefore be a central aspect of computing education. We have conducted a systematic mapping study to give a broad overview of the research conducted in the field of code quality in an educational context. The study investigates paper characteristics, topics, research methods, and the targeted programming languages. We found 195 publications (1976-2022) on the topic in multiple databases, which we systematically coded to answer the research questions. This paper reports on the results and identifies developments, trends, and new opportunities for research in the field of code quality in computing education

Exploring the Potential of Large Language Models to Generate Formative Programming Feedback

Ever since the emergence of large language models (LLMs) and related applications, such as ChatGPT, its performance and error analysis for programming tasks have been subject to research. In this work-in-progress paper, we explore the potential of such LLMs for computing educators and learners, as we analyze the feedback it generates to a given input containing program code. In particular, we aim at (1) exploring how an LLM like ChatGPT responds to students seeking help with their introductory programming tasks, and (2) identifying feedback types in its responses. To achieve these goals, we used students' programming sequences from a dataset gathered within a CS1 course as input for ChatGPT along with questions required to elicit feedback and correct solutions. The results show that ChatGPT performs reasonably well for some of the introductory programming tasks and student errors, which means that students can potentially benefit. However, educators should provide guidance on how to use the provided feedback, as it can contain misleading information for novices.Comment: Accepted to FIE 202

What Skills Do You Need When Developing Software Using ChatGPT? (Discussion Paper)

Since the release of LLM-based tools such as GitHub Copilot and ChatGPT the media and popular scientific literature, but also journals such as the Communications of the ACM, have been flooded with opinions how these tools will change programming. The opinions range from ``machines will program themselves'', to ``AI does not help programmers''. Of course, these statements are meant to to stir up a discussion, and should be taken with a grain of salt, but we argue that such unfounded statements are potentially harmful. Instead, we propose to investigate which skills are required to develop software using LLM-based tools. In this paper we report on an experiment in which we explore if Computational Thinking (CT) skills predict the ability to develop software using LLM-based tools. Our results show that the ability to develop software using LLM-based tools can indeed be predicted by the score on a CT assessment. There are many limitations to our experiment, and this paper is also a call to discuss how to approach, preferably experimentally, the question of which skills are required to develop software using LLM-based tools. We propose to rephrase this question to include by what kind of people/programmers, to develop what kind of software using what kind of LLM-based tools.Comment: 11 page

A Systematic Mapping Study of Code Quality in Education -- with Complete Bibliography

While functionality and correctness of code has traditionally been the main focus of computing educators, quality aspects of code are getting increasingly more attention. High-quality code contributes to the maintainability of software systems, and should therefore be a central aspect of computing education. We have conducted a systematic mapping study to give a broad overview of the research conducted in the field of code quality in an educational context. The study investigates paper characteristics, topics, research methods, and the targeted programming languages. We found 195 publications (1976-2022) on the topic in multiple databases, which we systematically coded to answer the research questions. This paper reports on the results and identifies developments, trends, and new opportunities for research in the field of code quality in computing education

Exploring the Potential of Large Language Models to Generate Formative Programming Feedback

Ever since the emergence of large language models (LLMs) and related applications, such as ChatGPT, its performance and error analysis for programming tasks have been subject to research. In this work-in-progress paper, we explore the potential of such LLMs for computing educators and learners, as we analyze the feedback it generates to a given input containing program code. In particular, we aim at (1) exploring how an LLM like ChatGPT responds to students seeking help with their introductory programming tasks, and (2) identifying feedback types in its responses. To achieve these goals, we used students' programming sequences from a dataset gathered within a CS1 course as input for ChatGPT along with questions required to elicit feedback and correct solutions. The results show that ChatGPT performs reasonably well for some of the introductory programming tasks and student errors, which means that students can potentially benefit. However, educators should provide guidance on how to use the provided feedback, as it can contain misleading information for novices

Code quality Issues in Student Programs

Because low quality code can cause serious problems in software systems, students learning to program should pay attention to code quality early. Although many studies have investigated mistakes that students make during programming, we do not know much about the quality of their code. This study examines the presence of quality issues related to program !ow, choice of programming constructs and functions, clarity of expressions, decomposition and modularization in a large set of student Java programs. We investigated which issues occur most frequently, if students are able to solve these issues over time and if the use of code analysis tools has an e"ect on issue occurrence. We found that students hardly #x issues, in particular issues related to modularization, and that the use of tooling does not have much e"ect on the occurrence of issues

A Systematic Mapping Study of Code Quality in Education

While functionality and correctness of code has traditionally been the main focus of computing educators, quality aspects of code are getting increasingly more attention. High-quality code contributes to the maintainability of software systems, and should therefore be a central aspect of computing education. We have conducted a systematic mapping study to give a broad overview of the research conducted in the field of code quality in an educational context. The study investigates paper characteristics, topics, research methods, and the targeted programming languages. We found 195 publications (1976-2022) on the topic in multiple databases, which we systematically coded to answer the research questions. This paper reports on the results and identifies developments, trends, and new opportunities for research in the field of code quality in computing education.</p

Strategy-based feedback in a programming tutor

More and more people take up learning how to program: in schools and universities, in large open online courses or by learning it by themselves. A large number of tools have been developed over the years to support learners with the difficult task of building programs. Many of these tools focus on the resulting program and not on the process: they fail to help the student to take the necessary steps towards the final program. We have developed a prototype of a programming tutor to help students with feedback and hints to progress towards a solution for an introductory imperative programming problem. We draw upon the ideas of a similar tutor for functional programming and translate these ideas to a different paradigm. Our tutor is based on model solutions from which a programming strategy is derived, capturing the different paths to these solutions. We allow for variation by expanding the strategy with alternatives and using program transformations. The instructor is able to adapt the behaviour of the tutor by annotating the model solutions. We show a tutoring session to demonstrate that a student can arrive at a solution by following the generated hints. We have found that we can recognise between 33% and 75% of student solutions to three programming exercises that are similar to a model solution, which we can increase by incorporating more variations

A Tutoring System to Learn Code Refactoring

In the last few decades, numerous tutoring systems and assessment tools have been developed to support students with learning programming, giving hints on correcting errors, showing which test cases do not succeed, and grading their overall solutions. The focus has been less on helping students write code with good style and quality. There are several professional tools that can help, but they are not targeted at novice programmers. This paper describes a tutoring system that lets students practice with improving small programs that are already functionally correct. The system is based on rules that are extracted from input by teachers collected in a preliminary study, a subset of rules taken from professional tools, and other literature. Rules define how a code construct can be rewritten into a better variant, without changing its functionality. Rules can be combined to form rewrite strategies, similar to refactorings offered by most IDEs. The student can ask for hints and feedback at each step. We describe the design of the system, show example sessions, and evaluate and discuss its contribution and limitations